1. Field of the Invention
The invention relates generally to virtualization and more specifically relates to Peripheral Component Interconnect Express (PCIe) technologies for enhancing Single Root Input/Output Virtualization (SR-IOV).
2. Discussion of Related Art
SR-IOV is a PCIe technology that is used to implement Input/Output (I/O) virtualization. In virtualized systems, one or more Virtual Machines (VMs) operate on a single physical computing system. Each VM operates on the computing system substantially independently of its peers. For example, each VM may have its own memory space on the physical computing system, and may implement its own Operating System (OS) and associated programs. However, the VMs share the physical resources of the computing system. Thus, one physical computer may be used to host multiple VMs, each performing a different function. This ensures that the processors of the physical computer are effectively utilized to service multiple tasks.
Sharing of physical resources between VMs is typically performed without the knowledge of the VMs. The individual VMs operate as if they have complete control of the physical computing system, and do not normally communicate with each other. Sharing of physical resources between VMs is performed by a Virtual Machine Manager (VMM) (also known as a hypervisor), which implements a hardware abstraction layer between the VMs and the physical resources of the system. Unbeknownst to the VMs, the VMM may prioritize, queue, or otherwise manage the order and manner in which I/O for the VMs is processed by the resources of the physical computing system. The VMM may also set up initial memory spaces for each VM, boot each VM onto the physical computing system, etc. As far as each VM is concerned, it has initialized in an environment where it has its own, dedicated physical computing hardware.
In I/O virtualization, a single I/O hardware device (such as a PCIe RAID adapter or network adapter) is shared between multiple VMs. The sharing can be performed using software (emulated), hardware (e.g., SR-IOV), or hybrid (Paravirtualization) sharing techniques.
FIG. 1 illustrates an exemplary SR-IOV system 100, where a physical computing system 120, such as a host, sends I/O to a physical I/O device 110. Physical device 110 uses SR-IOV to implement Physical Function (PF) 112 at a physical I/O device 110, and also to implement multiple Virtual Functions (VFs) 114 at the physical I/O device. PF 112 is hardware circuitry that acts as a standard interface for I/O device 110. PF 112 communicates with a physical function driver 123 at the emulation layer of VMM 122. PF 112 also has access to hardware registers that implement the full PCI configuration space for physical I/O device 110.
In contrast to PF 112, each VF 114 is implemented in firmware as a processor implementing logical instructions. Furthermore, each VF 114 communicates with a virtual function driver 125 of a single VM 124. The virtual function drivers 125 bypass an emulation layer in VMM 122, providing VMs 124 direct access to the VFs 114 of I/O device 110. VFs 114 at physical I/O device 110 receive I/O from virtual function drivers 125. Each VF 114 acts as an independent, virtual port of I/O device 110, and uses hardware registers to implement a limited subset of the PCIe configuration space of physical I/O device 110. This enhances processing quality at control block 116, which performs core control functions for I/O device 110.
The VMM assigns one or more VFs to a VM by mapping configuration space registers of the VFs to the configuration space presented to the VM by the VMM. SR-IOV-capable devices can provide configurable numbers of independent VFs, each with its own PCIe configuration space.
Each VF implements a subset of traditional PCIe configuration space hardware registers, and accesses the remaining configuration space registers via the associated PF. Specifically, a Single Root PCI Manager (SR-PCIM) can intercept attempts by VMs to access configuration space registers that are not a part of the VF, and can use the information from the PF registers to present a complete PCIe configuration space to a guest OS running in a VM.
Unfortunately, when I/O device 110 is first powered on, VFs 114 do not exist inside VMM 222 and they cannot retrieve their hardware registers for the PCIe configuration space. This is also true even for registers that are accessed by VFs 114 via PF 112, because the SR-PCIM (existing in software) will not be available to acquire these registers from PF 112 while the system is still booting. Thus, when a physical I/O device is rebooted, the configuration space of each VF 114 is not available. This can cause errors at each VM 124 that attempts to contact a VF 114, because the VF 114 becomes completely unresponsive to the VM 124. The VM 124 is unable to determine the reason for the lack of responsiveness.
Thus it is an ongoing challenge to enhance interactions between VMs and VFs in a PCI SR-IOV environment.